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Abstract-  A  novel  sea  mine  computer-aided-detection  / 
computer-aided-classification  (CAD/CAC)  processing  string  has 
been  developed.  The  overall  CAD/CAC  processing  string 
consists  of  pre-processing,  adaptive  clutter  filtering  (ACF), 
normalization,  detection,  feature  extraction,  feature 
orthogonalization,  optimal  subset  feature  selection,  classification 
and  fusion  processing  blocks.  The  range-dimension  ACF  is 
matched  both  to  average  mine  highlight  and  shadow 
information,  while  also  adaptively  suppressing  background 
clutter.  For  each  detected  object,  features  are  extracted  and 
processed  through  an  orthogonalization  transformation, 
enabling  an  efficient  application  of  the  optimal  log-likelihood- 
ratio-test  (LLRT)  classification  rule,  in  the  orthogonal  feature 
space  domain.  The  classified  objects  of  4  distinct  processing 
strings  are  fused  using  the  classification  confidence  values  as 
features  and  “M-out-of-N”,  or  LLRT-based  fusion  rules.  The 
utility  of  the  overall  processing  strings  and  their  fusion  was 
demonstrated  with  new  shallow  water  high-resolution  sonar 
imagery  data.  The  processing  string  detection  and  classification 
parameters  were  tuned  and  the  string  classification  performance 
was  optimized,  by  appropriately  selecting  a  subset  of  the  original 
feature  set.  Two  significant  improvements  were  made  to  the 
CAD/CAC  processing  string  by  employing  sub-image  adaptive 
clutter  filtering  (SACF)  and  utilizing  a  repeated  application  of 
the  subset  feature  selection  /  LLRT  classification  blocks.  It  was 
shown  that  LLRT-based  fusion  of  the  CAD/CAC  processing 
strings  outperforms  the  “M-out-of-N”  algorithms  and  results  in 
up  to  an  eight-fold  false  alarm  rate  reduction,  compared  to  the 
best  single  CAD/CAC  processing  string  results,  while 
maintaining  a  constant  correct  mine  classification  probability. 

I.  INTRODUCTION 

The  long-term  goal  of  this  project  is  to  develop  robust, 
automated  sea  mine  detection,  classification  and  fusion 
algorithms  to  be  utilized  by  imaging  sonars  on-board  of 
unmanned  underwater  vehicles  (UUV)  operating  in  shallow 
water  (SW)  or  very  shallow  water  (VSW)  environments.  This 
technology  will  help  enable  the  mine-countermeasures 
(MCM)  vehicle  operation  in  the  VSW,  thus  substituting  Navy 
divers  and  marine  mammals  that  are  currently  performing  this 
extremely  hazardous  task. 

The  near-term  objective  of  this  project  is  to  demonstrate 
automated  sea  mine  detection  and  classification  in  the  VSW, 
using  CAD/CAC  and  fusion  algorithms  that  are  being 
developed  by  4  research  teams  at  the  Naval  Surface  Warfare 
Center  (NSWC)  Coastal  Systems  Station  (CSS),  Raytheon, 
Alphatech,  and  Lockheed  Martin  (LM).  This  work  is  in 
support  of  mine-hunting  demonstrations  for  ONR's  Ocean 
Engineering  and  Marine  Systems  programs. 

This  paper  builds  on  LM’s  previous  work  [l]-[9]  in  the 
automated  detection  and  classification  of  sea  mines  in  sonar 
imagery.  It  discusses  two  significant  CAD/CAC  processing 
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string  improvements:  the  first  entails  utilizing  a  sub-image 
adaptive  clutter  filter  instead  of  the  single  adaptive  filter  per 
image  that  was  previously  employed,  whereas  the  second 
entails  the  repeated  application  of  the  subset  feature  selection, 
feature  orthogonalization  and  LLRT  classification  processing 
blocks  in  a  cascade  fashion  [10],  [11].  This  latter  approach 
was  first  utilized  by  Dobeck  [12]  and  was  referred  to  as  a 
“classifier  anding”  procedure. 

The  overall  system  performance  can  of  course  be 
significantly  enhanced  by  fusing  the  outputs  of  distinct 
processing  strings.  In  recent  papers  the  authors  presented  the 
results  of  fusing  the  outputs  of  3  processing  strings  and 
utilizing  a  VSW  dataset  [6]-[9],  or  a  SW  dataset  [10].  The 
authors  most  recent  paper  presented  the  results  of  fusing  4 
distinct  processing  strings  and  utilizing  a  new  SW  dataset 
[11],  This  paper  presents  the  results  of  fusing  the  improved 
LM  CAD/CAC  processing  string  with  the  3  other  ones  and 
utilizing  the  same  latest  SW  dataset.  The  fusion  algorithms 
evaluated  here  include  “M-out-of-N”  and  feature 
orthogonalization  /  LLRT-based  algorithms. 

II.  CAD/CAC  STRING  DESCRIPTION  AND 
PROCESSING  RESULTS 

A.  Baseline  CAD/CAC  Processing  String 

An  automatic,  robust,  adaptive  clutter  suppression,  sea 
mine  detection  and  classification  processing  string  has  been 
developed  and  applied  to  high-resolution  sonar  imagery  data. 
Fig.  1  shows  this  baseline  CAD/CAC  processing  string,  which 
includes  pre-processing,  adaptive  clutter  filtering  (ACF), 
normalization  and  detection,  feature  extraction  and 
orthogonalization,  best  subset  feature  selection  and  LLRT 
classification  processing  blocks. 

The  pre-processing  block  contains  pre-normalization, 
clipping  and  data  decimation  blocks.  Normalization  reduces 
data  non-homogeneity.  A  combination  feed-forward  and 
backward  normalizer  was  employed,  which  computes  water 
column  information  and  was  developed  by  Dobeck  [13]. 
After  normalization,  data  clipping  better  conditions  the  data 
for  ACF  processing,  by  resulting  in  a  more  stable  target 
signature  for  the  algorithm  design  and  application. 
Subsequently  applied  data  decimation  reduces  the  overall 
processing  string  implementation  complexity  and  again 
facilitates  ACF  design  and  application. 

The  ACF  is  an  adaptive  linear  FIR  filter,  which  is 
optimal  in  the  Least  Squares  (LS)  sense  and  is  applied  to  low- 
resolution  data.  It  performs  simultaneous  background  clutter 
suppression  and  peak  target  preservation  by  exploiting 
differences  between  clutter  and  target  correlation 
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Fig.  1.  Baseline  mine  detection  and  classification  processing  string. 


characteristics.  In  the  latest  algorithm  version,  a  1- 
dimensional  range-only  ACF,  that  was  matched  to  both 
average  highlight  and  shadow  target  shape  information,  was 
applied  after  mean  removal.  A  wide-sense  stationary 
covariance  [5]  model  is  utilized  in  the  algorithm  design, 
which  significantly  reduces  the  algorithm  implementation 
complexity,  thus  enabling  an  easy  implementation  of  the 
overall  processing  string  in  real-time. 

The  ACF  block  output  is  processed  through  a  crossrange 
domain,  whole  column,  global  normalizer,  which  employs 
negative  and  small  value  data  clipping  and  removes  any 
remaining  non-stationarities  in  the  data.  Subsequently, 
detection  processing  is  applied,  consisting  of  thresholding, 
clustering  of  exceedances  (based  on  the  maximum  expected 
mine  target  size),  application  of  secondary  thresholding 
(discarding  all  detections  that  are  point-like)  and  limiting  the 
number  of  detections. 

Referring  to  Fig.  1,  features  or  clues  are  extracted  for 
each  detected  contact,  using  high-resolution  input  sonar  data, 
centered  at  the  corresponding  local  region.  Subsequently,  the 
features  are  processed  through  an  orthogonalization 
transform,  which  enables  an  efficient  application  of  the 
optimal  LLRT  classification  rule  in  the  orthogonal  feature 
space  domain.  Another  important  point  is  the  utilization  of 
software  tools  for  the  selection  of  feature  and  LLRT  classifier 
parameters  and  the  selection  of  a  subset  of  the  features  ( “the 
best  features”)  using  a  training  set,  and  thus  optimizing  the 
processing  string  classification  performance. 

B.  Improved  CAD/CAC  Processing  String 

1 )  Sub-image  Adaptive  Clutter  Filter  (SACF) 

The  baseline  CAD/CAC  processing  string  utilized  a 
single  ACF  per  image.  This  algorithm  computed  the 
background  covariance  statistics  over  the  whole  image  and 
was  matched  to  a  signature  vector  obtained  by  averaging  over 
all  the  training  set  mine  signatures.  The  baseline  ACF 
algorithm  performed  well,  since  typically  the  images  inputted 
into  the  ACF  are  rather  homogeneous,  since  they  have  already 
been  pre -normalized.  On  the  other  hand,  a  modified 
algorithm  that  breaks  the  image  into  sub-images  and 
computes  a  different  ACF  for  each  sub-image  offers 
advantages,  since  it  can  better  adapt  to  the  covariance 
statistics  of  each  sub-image  and  to  the  signature  variations  in 
each  sub-image  (e.g.  mines  at  longer  ranges  are  known  to 
have  longer  shadows). 

The  optimum  filter  formulation  of  the  SACF  algorithm 
involves  the  minimization  of  the  cost  function,  for  the  ith  sub¬ 
image  of 


I,  =  0.5  WiT  R,  Wj  -  X  (  WjT  S;  -  1  )  -  p  ( wT  1  -  0 ) ,  (1) 

where  w;  is  the  adaptive  filter  weight  vector,  w,1  denotes 
vector  transpose,  R,  is  the  clutter  covariance  matrix,  Sj  is  the 
average  target  signal  signature  within  the  filter  window,  1  is  a 
constant  vector  whose  components  are  all  equal  to  1  and 
X  and  p  are  Language  multipliers.  The  first  term  in  (1)  refers 
to  clutter  minimization,  the  second  term  is  a  target 
preservation  constraint  and  the  third  term  is  a  mean  removal 
constraint,  such  that  there  are  no  mean  mismatches  after 
application  of  the  SACF  algorithm  to  different  sub-images 
( this  is  necessary  since  post  ACF  processing  involves  further 
image  normalization). 

Minimization  of  (1)  results  in  the  normal  equations 
R;  w,  =  A,  s,  +  p  1 ,  (2) 

which  are  solved  for  the  optimum  adaptive  linear  filter  w,. 

In  the  SACF  procedure,  for  each  sub-image  at  each 
spatial  location  the  1 -dimensional  range-only  search-box 
contents  are  scanned  into  a  local  vector  Xi(k)  and  the 
corresponding  snapshot  of  the  local  covariance  R;(k)  is 
computed.  Sliding  the  search-box  over  the  whole  sub-image 
and  adding  all  the  local  covariances,  the  overall  accumulated 
sub-image  covariance  R;  is  then  computed  as 


Substituting  (3)  and  the  average  target  signature  vector  Sj 
into  (2)  and  solving,  using  a  Cholesky  decomposition  of  the 
covariance  matrix  followed  by  back  substitutions,  yields  the 
optimal  adaptive  spatial  filter  W; 

wi  =  Ri'1Ai/(AiTRi-1Ai)b,  (4) 

where  matrix  A;  and  vector  b  are  defined  as 

Ai  =  [  Si  1  ] , 

bT=[l  0]. 

Finally,  the  data  is  filtered  as 

y,(k)  =  w;T  Xj(k) .  (5) 

The  SACF  procedure  was  outlined  by  (l)-(5).  Typically, 
an  average  target  signature  vector  Sj  is  estimated  from  all  the 
training  set  mine  signature  data  in  the  ith  sub-image.  For  each 


1112 


target  in  the  training  set  the  1 -dimensional  filter  window  is 
centered  at  the  target  mask,  with  the  largest  value  normalized 
to  1.  The  contents  of  the  search-box  are  then  scanned  into  a 
vector  to  compute  the  normalized  signature  vector  sik  for  the 
kth  target.  Subsequently,  an  average  signature  is  computed  by 
averaging  the  signatures  of  all  the  members  belonging  to  the 
training  class  as 

M 

s,-  =  m£s«-  •  (6) 

k= 1 

2)  Cascade  Classification 

Each  dataset  is  typically  split  into  training  and  test  sets, 
each  set  containing  roughly  half  the  images  and  half  the 
targets.  The  classifier  is  trained  on  the  training  set  only,  to 
help  avoid  over-training  the  classifier,  but  performance  is 
optimized  over  the  whole  dataset.  Cascade  classification 
entails  repeated  application  of  the  classification  stage.  After 
the  first  stage,  classifier  parameters  are  selected  in  such 
fashion  that  all  targets  are  correctly  called,  with  each 
classification  stage  discarding  as  many  false  alarms  as 
possible. 

With  LM’s  cascade  classifier  design,  the  subset  feature 
selection,  feature  orthogonalization  and  LLRT  classification 
process  is  repeated  until  classification  performance  has 
stabilized.  In  each  stage,  the  best  subset  of  features,  the 
appropriate  number  of  LLRT  histogram  bins  and  the 
classification  threshold  are  selected  to  optimize  the 
corresponding  classification  performance.  Each  progressing 
stage  results  in  a  diminishing  false  alarm  feature  set,  whereas 
the  target  training  and  test  feature  sets  remain  fixed. 

The  cascade  classification  technique  is  illustrated  in  Lig. 
2.  During  the  training  phase  of  the  algorithm,  it  is  determined 
how  many  times  to  apply  the  classification  process  (typically 
3-4  times),  and  each  stage  determines  and  stores  the 
classification  threshold,  the  best  subset  of  features,  the 
corresponding  orthogonalization  transformation  matrix,  the 
number  and  boundaries  of  the  histogram  bins  and  the 
corresponding  LLRT  values  in  the  orthogonal  feature  space 
domain. 

In  the  testing  phase  of  the  algorithms  all  these  parameters 
are  fixed  and  are  looked  up  from  a  library.  The  performance 
stabilizer  block  becomes  then  a  counter  and  a  memory  storage 
pointer  of  the  classification  stage  parameters.  After 
processing  feature  data  3-4  times,  the  final  classified  objects 
are  fed  to  the  output  stage.  The  improved  CAD/CAC 
processing  string  entails  replacing  the  classification  stage  of 
Lig.  1  (the  3  right  hand  blocks)  with  Lig.  2. 


C.  Processing  String  Performance  Evaluation  Results 

Lockheed  Martin  evaluated  the  ACL,  detection  and 
classification  algorithm  performance  with  a  new  shallow 
water  sonar  data  set,  collected  off  the  coast  of  Southern 
California  and  in  the  Gulf  of  Mexico  in  2001.  The  dataset 
consisted  of  113  images,  containing  a  total  of  86  targets  [11], 
The  data  was  split  into  a  training  set  (containing  half  the 
images  and  half  the  targets)  and  a  test  set,  both  of  which  were 
utilized  for  the  algorithm  performance  evaluation  and  testing 
phase. 

The  sonar  image  data  was  first  processed  through  the 
baseline  Lockheed  Martin  (LM)  CAD/CAC  processing  string 
of  Lig.  1.  The  pre-processing  block  included:  pre¬ 
normalization  and  clipping  the  sonar  images;  averaging  and 
employing  a  2:1  data  decimation  factor  in  both  range  and 
crossrange  dimensions,  and  applying  mean  removal.  Data 
was  subsequently  processed  through  a  range -only  31 -tap  ACL 
that  was  matched  both  to  average  highlight  and  shadow  shape 
information,  and  a  global  post-ACL  normalizer,  with  negative 
data  and  noise  level  clipping.  Subsequently  data  was 
processed  through  the  detection  block,  and  the  detection 
parameters  were  optimized. 

After  detection  processing,  high-resolution  clues  were 
extracted  from  the  input  sonar  data.  Originally,  each  detected 
object  was  characterized  by  an  1 1 -dimensional  feature  vector, 
described  in  Lig.  3.  The  thresholds  and  window  sizes  of  the 
feature  parameters  were  selected  to  optimize  the  LLRT 
classifier  performance.  The  classifier  was  trained  on  the 
training  set.  Performance  evaluation  results  were  with  the 
whole  dataset. 

The  overall  processing  string  mine  classification 
performance  was  optimized  by  selecting  a  subset  of  the 
features,  “the  best  features”.  The  baseline  LM  CAD/CAC 
string  performance  with  the  SW  dataset,  when  using  the  “7 
best  features”,  was  0.73  average  false  alarms/image  (LAPI), 
while  correctly  classifying  78  out  of  86  targets,  P(|PC=0.907 
[11]. 

The  baseline  LM  CAD/CAC  processing  string  used  a 
single  ACL  per  image  algorithm.  The  first  significant 
improvement  of  the  processing  string,  described  in  Section 
II.B.l,  entailed  splitting  the  image  into  sub-images  and 
designing  sub-image  adaptive  clutter  filters.  A  preliminary 
investigation  of  this  SACL  algorithm  showed  that  splitting  the 
image  into  3  sub-images,  and  designing  and  applying  3 
SACLs,  resulted  in  a  significant  performance  improvement. 
The  improved  SACL-based  CAD/CAC  string  performance 
was  LAPI=0.67,  while  correctly  classifying  80  out  of  86 
targets,  P(jPc=0.93.  The  baseline  ACL  and  improved  SACL 
CAD/CAC  string  results  are  summarized  in  Table  1. 


Fig.  2.  Cascade  classification  technique. 
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Feature  Listing 

Selected  Feature  at 
Classifier  Stage  # 

1 

2 

3 

1.  Peak  Magnitude 

☆ 

☆ 

2.  #  Highlight  Pixels  in  5x5  Window 

☆ 

☆ 

3.  #  Highlight  Pixels  in  a  LxXL2  Wind. 

☆ 

☆ 

☆ 

4.  Normalized  Variance 

☆ 

☆ 

☆ 

5.  Skew 

☆ 

☆ 

6.  Kurtosis 

☆ 

☆ 

7.  Length 

☆ 

8.  Width 

☆ 

☆ 

9.  Orientation 

☆ 

10.  Shadow  Pixel  Count 

☆ 

☆ 

11.  Shadow  Length 

☆ 

☆ 

#  of  Selected  Features 

8 

8 

6 

Fig.  3.  Feature  listing  and  selected  features 
(denoted  by  ★ )  at  each  classification  stage. 


The  2nd  improvement  of  the  CAD/CAC  processing  string 
of  Section  II. B. 2,  uses  the  multistage  classifier  of  Fig.  2.  Its 
mine  classification  performance  with  the  SW  dataset  is 
summarized  in  Fig.  4.  Each  classification  stage  reduces  false 
alarms,  while  maintaining  a  constant  mine  classification 
probability  of  PcjPc=0.93.  Processing  data  through  a  multi¬ 
stage  classifier  3-times  results  in  a  FAPI=0. 177,  which 
corresponds  to  a  3.6:1  false  alarm  reduction  vs.  the  single- 
stage  classifier  performance.  Referring  to  Fig.  4,  it  is  noted 
that  false  alarm  reduction  performance  has  nearly-stabilized 
after  the  3th  stage,  i.e.  a  small  additional  false  alarm  reduction 
is  provided  by  the  3th  stage,  which  explains  why  it  was  chosen 
to  process  data  3  times  through  our  multi-stage  classifier. 
Also,  notice  that  the  early  classification  stages  provide  a  more 
significant  false  alarm  reduction  than  the  later  ones. 
Additionally,  for  each  classification  stage,  we  select  an 
optimal  feature  subset  as  shown  in  Fig.  3. 

It  should  be  noted  that,  with  each  new  dataset  there  are 
always  CAD/CAC  processing  string  parameters  that  need  to 
be  determined,  to  optimize  detection  and  classification  string 
performance.  These  include:  noise  clipping  factor,  SACF 
reference  signatures,  detection  and  classification  thresholds, 
feature  window  parameters  and  multi-stage  classifier 
parameters,  including  classification  thresholds,  number  of 
histogram  bins,  subsets  of  best  features,  LLRT  histogram 
values  and  orthogonalization  transformation  matrices,  all 
determined  during  the  training  phase  of  the  algorithms. 

III.  CAD/CAC  STRING  FUSION  RESULTS 
A.  Fusion  Approach 

The  authors  approach  in  this  paper  is  to  fuse  the  results 
of  4  drastically  different  CAD/CAC  processing  strings  [11], 
The  1st  CAD/CAC  processing  string  was  developed  by 
Lockheed  Martin-Syracuse  (Tom  Aridgides  and  Manuel 


TABLE  1 .  BASELINE  ACF  AND  IMPROVED  SACF 
CAD/CAC  STRING  PERFORMANCE  SUMMARY 


CAD/CAC  String 

PdPc 

FAPI 

Improved  SACF-based 

0.930 

0.63 

Baseline  ACF-based 

0.907 

0.73 

Fernandez  [1]-[11]),  the  2nd  was  developed  by  CSS  (Gerald  J. 
Dobeck  [12]-[16]),  the  3rd  was  developed  by  Raytheon 
(Charles  Ciany,  Jim  Huang,  et.  al.  [17]),  whereas  the  4lh  was 
developed  by  Alphatech  (Marty  Bello  [18]),  all  under  ONR 
sponsorship. 

The  fusion  of  algorithms  is  a  very  powerful  tool,  since 
each  algorithm  is  looking  at  the  same  data  from  a  different 
perspective.  This  is  due  to  the  fact  that  each  algorithm  is 
based  on  different  mathematical,  statistical,  and  geometric 
theories,  and  thereby  emphasizes  (or  de -emphasizes)  different 
characteristics  of  the  data  (clutter  and  targets).  Therefore 
much  can  be  gained  through  the  algorithm  fusion,  especially 
in  false  alarm  reduction,  where  typically  few  false  alarms  are 
common  to  all  algorithms. 

The  outputs  of  the  Lockheed  Martin,  CSS,  Raytheon  and 
Alphatech  CAD/CAC  strings  were  fused  as  shown  in  Fig.  5. 
Calls  by  different  algorithms  were  grouped  into  the  same  call, 
based  on  distance  criteria  derived  from  the  expected  sizes  of 
targets  and  also  accounting  for  differences  of  the  various 
algorithms  in  computing  the  centroid  of  the  target.  The 
authors  of  this  paper  utilized  a  variety  of  fusion  algorithms, 
including:  “M-out-of-N”,  and  orthogonal  LLRT-based.  The 
confidences  of  the  4  CAD/CAC  string  outputs  were  used  as 
features  for  the  orthogonal  LLRT-based  algorithm. 

B.  Fusion  Results 

The  113  SW  sonar  images  were  processed  through  the 
tuned  and  improved  Lockheed  Martin,  CSS,  Raytheon  and 
Alphatech  CAD/CAC  processing  strings.  The  single  string 
CAD/CAC  processing  results  are  summarized  in  Table  2.  In 
Table  2,  Ndet  denotes  the  number  of  mine  targets  correctly 
called  by  an  algorithm,  PdPc  denotes  the  corresponding 
probability  of  correct  mine  classification,  nfa  denotes  the 
number  of  false  alarms  and  FAPI  denotes  the  average  number 
of  false  alarms  per  image. 

Each  string  output  j  (j=  1 ,2,3,4)  generated  a  number  of 
calls,  indicating  the  corresponding  image  file,  the  call  location 
X(j,  Yy  and  a  confidence  factor  Clp  associated  with  each  call  i. 

For  each  image  file,  a  call  i  by  algorithm  j  and  a  call  m 
by  algorithm  n  are  attributed  to  the  same  object,  iff: 

dis  {  (  Xij ,  Yy) ,  (  X„  ,  Ymn  )  }  <  diSmin  ,  (7) 

where  dismm  is  based  on  the  largest  mine  size.  If  algorithm  j 
has  detected  an  object  i,  then  the  corresponding  confidence 
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TABLE  2.  SINGLE  CAD/CAC  STRING  PERFORMANCE  SUMMARY 


Fig.  5.  CAD/CAC  string  fusion. 


value  Qj  is  such  that, 

0.5  <  Cy  <  1  ,  (8) 

(where  Cy  =  0.5  denotes  50%  confidence  about  the 
classification  decision  and  Cy  =  1  denotes  100%  confidence). 

After  registration  of  calls  by  different  algorithms,  each 
call  is  characterized  by  a  4-dimensional  feature  vector 

Ci=[Cil  Ci2  Ci3  Ci4],  (9) 

where  Qj=0  denotes  that  algorithm  j  did  not  classify  this 
object. 

Lockheed  Martin  has  developed  and  refined  “M-out-of- 
N”  and  orthogonal  LLRT-based  algorithms  to  fuse  the  data 
and  has  analyzed  the  fusion  algorithm  performance.  The 
baseline  “M-out-of-N”  algorithms  do  not  use  actual 
confidence  values,  but  LLRT-based  ones  do. 

1 )  M  out  ofN  Fusion  Results 


CAD/CAC  String 

Ndet 

PdPc 

nfa 

FAPI 

Alg.  1 

80 

0.930 

20 

0.177 

Alg.  2 

78 

0.907 

41 

0.363 

Alg.  3 

77 

0.895 

69 

0.611 

Alg.  4 

74 

0.860 

69 

0.611 

•  “Modified  3-out-of-4”  (a  fusion  call  is  declared  iff  it 

was  made  by  at  least  3  out  of  4  algorithms  and  the  sum 
of  the  confidence  values  was  greater  than  1.803,  i.e. 
1.803  <  E  Cy  ,  this  threshold  value  was  also 

determined  from  the  data,  by  looking  at  all  fused  target 
calls  and  determining  the  minimum  aggregate 
confidence  value). 

•  “4-out-of-4”  (a  fusion  call  is  declared  iff  it  was  made  by 
all  4  algorithms  -  this  is  the  same  as  an  ANDing  rule). 
The  “M-out-of-N”  fusion  results  are  summarized  in 

Table  3.  Comparing  these  results  to  the  individual  CAD/CAC 

string  results  of  Table  2,  the  following  fusion  performance 

processing  improvements  are  noted: 

•  “Alg.  N+l”  fusion  provides  between  2.3:1  and  3.1:1 
false  alarm  reduction  over  the  CAD/CAC  string  N 
alone,  when  maintaining  the  same  PdPc. 

•  “1  out  of  4”  algorithm  fusion  provides  the  capability  to 
correctly  call  all  targets,  but  at  a  cost  of  a  high  false 
alarm  rate,  FAPI=  1 .4. 

•  “2-out-of-4”  fusion  provides  a  high  mine  classification 
capability,  PdPc=0.942,  with  only  FAPI=0.283. 

•  “Modified  2-out-of-4”  fusion  provides  an  additional 
1.3:1  false  alarm  reduction,  over  “2-out-of-4”  fusion. 

•  “3-out-of-4”  and  “modified  3-out-of-4”  fusion  both 
result  in  a  very  low  false  alarm  rate,  FAPI=0.062,  at 
PdPc=0.895. 

•  “4  out  of  4”  algorithm  fusion  results  in  0  false  alarms, 
but  at  a  cost  of  low  PdPc=0.744. 


The  authors  employed  the  commonly  utilized  “M-out-of- 
N”  fusion  rules  to  combine  the  individual  CAD/CAC  string 
outputs.  Calls  were  grouped  together  prior  to  the  application 
of  the  various  fusion  rules.  The  following  rules  were  utilized: 

•  “Alg.  N  +1”  (a  fusion  call  is  declared  iff  it  was  made  by 
algorithm  N,  where  N=l,  ...,4,  and  at  least  1  more  other 
algorithm). 

•  “l-out-of-4”  (a  fusion  call  is  declared  iff  it  was  made  by 
at  least  1  out  of  4  algorithms  -  this  is  the  same  as  an 
ORing  rule). 

•  “2-out-of-4”  (a  fusion  call  is  declared  iff  it  was  made  by 
at  least  2  out  of  4  algorithms). 

•  “Modified  2-out-of-4”  (a  fusion  call  is  declared  iff  it 
was  made  by  at  least  2  out  of  4  algorithms  and  the  sum 
of  the  confidence  values  was  greater  than  1.298,  i.e. 

1 .298  <  E  Cij  ,  this  threshold  value  was 

determined  from  the  data,  by  looking  at  all  fused  target 
calls  and  determining  the  minimum  aggregate 
confidence  value). 

•  “3-out-of-4”  (a  fusion  call  is  declared  iff  it  was  made  by 
at  least  3  out  of  4  algorithms). 


2 )  LLRT-based  Fusion  Algorithm  Results 

The  orthogonal  LLRT  fusion  algorithm  was  developed 
by  extending  the  Lockheed  Martin  feature  orthogonalization  / 
LLRT  classification  algorithm,  originally  developed  by 
Manuel  Fernandez  [19],  to  perform  fusion.  The  confidences 
of  the  4  CAD/CAC  string  outputs  were  combined  to  form  a  4- 
dimensional  vector,  as  shown  in  (9),  which  was  utilized  by  the 
feature  orthogonalization  /  LLRT  fusion  algorithm.  If  a 
particular  call  was  not  detected  by  a  given  CAD/CAC  string. 


TABLE  3.  M-OUT-OF-N  CAD/CAC  FUSION  SUMMARY 


CAD/CAC  String 

PdPc 

FAPI 

Alg.  1  +  1 

0.930 

0.071 

Alg.  2  +  1 

0.907 

0.159 

Alg.  3  +  1 

0.895 

0.195 

Alg.  4  +  1 

0.860 

0.204 

1  out  of  4 

1 

1.400 

2  out  of  4 

0.942 

0.283 

mod.  2  out  of  4 

0.942 

0.212 

3  out  of  4 

0.895 

0.062 

mod.  3  out  of  4 

0.895 

0.062 

4  out  of  4 

0.744 

0 
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the  feature  vector  value  at  the  corresponding  location  was 
originally  arbitrarily  set  to  zero  [6], 

The  feature  orthogonalization  /  LLRT  fusion  algorithm 
block  diagram  was  shown  in  [11]  (Fig.  7).  In  the  training 
phase  of  the  algorithm  the  learning  vectors  are  processed 
through  a  feature  orthogonalization  and  matrix  transformation 
extraction  procedure.  This  stage  can  be  used  to  remove 
redundant  information  and  reduce  the  dimensionality  of  the 
orthogonal  feature  vector  set.  Histograms  are  formed  for 
each  orthogonal  feature  and  the  LLRT  values  are  computed 
and  stored  in  a  library. 

In  the  testing  stage  of  the  procedure,  each  test  vector  is 
transformed  via  application  of  a  transformation  matrix,  and 
the  histogram  libraries  are  used  to  look-up  the  log-likelihood- 
ratio  values;  these  are  then  summed  to  obtain  the  overall 
composite  LLRT  value.  The  advantages  of  the  fusion 
algorithm  described  include:  it  enables  application  of  an 
optimal  decision  rule  (i.e.  the  LLRT),  entails  very  low 
complexity,  and  enhances  visualization  of  the  utility  of  each 
transformed  feature. 

A  new  investigation  was  presented  in  [7],  i.e.  if  no 
detection,  what  to  set  the  corresponding  confidence  value  to. 
It  was  shown  that  for  the  case  of  no  detection,  setting  the 
corresponding  confidence  value  to  Qj  =  0.125,  0.25  or  0.375 
provides  a  small,  but  noticeable  performance  improvement, 
over  the  previously  utilized  case  [6]  of  Qj  =0.  This 
investigative  work  was  repeated  with  the  SW  dataset  and 
similar  results  were  obtained. 

The  best  fusion  algorithm  performance  was  obtained  by 
utilizing  the  orthogonal  LLRT  fusion  algorithm.  The  LLRT- 
based  algorithm  fusion  results,  for  the  case  when  the 
algorithm  was  trained  on  all  data,  are  summarized  in  Fig.  6. 
The  advantages  that  fusion  provides  over  any  individual 
CAD/CAC  string  are  obvious  (e.g.  8:1  false  alarm  reduction 
over  the  best  performing  single  CAD/CAC  string,  while 
maintaining  the  same  probability  of  correct  target 
classification).  Alternatively,  fusion  can  drastically  increase 
the  correct  target  classification  probability,  as  compared  to 
the  best  single  CAD/CAC  string  performance  (i.e.  fusion 
provides  PdPc=0.988  at  a  FAPI  =  0.257,  whereas  the  Alg.  1 
string  results  in  PdPc  =  0.93  at  FAPI  =  0.177).  Thus,  thanks 
to  fusion  one  was  able  to  correctly  call  85  out  of  86  targets, 
whereas  the  Alg.  1  string  only  correctly  classified  8 1  of  them, 
but  with  fusion  we  had  the  cost  of  a  small  increase  in  false 
alarms. 


Fig.  7  summarizes  the  fusion  results  for  the  main 
approaches  investigated:  majority- voting  and  LLRT-based 
fusion.  It  is  noted  that  LLRT-based  fusion  (when  training  on 
all  data )  provides  the  best  results,  providing  an  additional 
11:1  false  alarm  reduction  over  “modified  2-out-of-4”  fusion, 
while  maintaining  a  constant  mine  classification  probability  of 
PdPc=0.942. 

In  the  results  just  presented  the  fusion  algorithm  was 
trained  on  all  the  data  and  was  tested  on  the  same  data.  Some 
performance  degradation  should  be  expected  if  the  algorithm 
is  trained  on  a  training  set  only  and  tested  on  all  the  data.  In 
general,  this  is  the  preferred  approach,  since  it  results  in  a 
more  robust  design,  and  it  will  be  the  one  employed  when 
designing  the  algorithm  parameters  for  future  planned  real¬ 
time  demonstrations. 

In  previous  work  [9]-[  10]  the  authors  had  investigated 
the  LLRT-based  fusion  algorithm  performance,  for  the  case 
when  the  algorithm  was  trained  on  training  set  data  only  and 
on  test  set  data  only.  In  this  case,  the  fusion  algorithm, 
experienced  a  2:1  increased  false  alarm  performance 
degradation,  as  compared  to  the  case  where  the  algorithm  was 
trained  on  all  the  data.  Nevertheless,  referring  to  Fig.  7,  even 
if  the  LLRT-based  fusion  algorithm  experienced  a  2:1 
increased  false  alarms  performance  degradation,  it  would  still 
significantly  outperform  the  “M-out-of-N”  algorithms. 
Therefore,  the  LRRT-based  algorithm  is  the  recommended 
CAD/CAC  string  fusion  approach. 

This  section  has  shown  the  performance  gains  provided 
by  using  LLRT-based  fusion  to  combine  the  outputs  of  4 
distinct  CAD/CAC  processing  strings.  This  raises  the 
question  whether  one  needs  to  use  all  4  CAD/CAC  strings 
and  what  performance  will  be  obtained  if  one  uses  only  3  or 
even  only  2  processing  strings.  The  authors  therefore  utilized 
the  LLRT-based  procedure  to  fuse  the  CAD/CAC  strings  of 
algorithms  1  and  2  (the  top  2  performing  algorithms)  and  to 
fuse  the  CAD/CAC  strings  of  algorithms  1,  2  and  3  (the  top  3 
performing  algorithms).  The  results  are  shown  in  Fig.  8.  It  is 
noted  that  using  LLRT-based  fusion  of  2  algorithms  only, 
provides: 

•  A  1.7:1  false  alarm  reduction  over  the  best  performing 
string  (algorithm  1),  at  PdPc=0.93. 

•  The  capability  to  detect  84  out  of  86  targets,  at  a  low 
false  alarm  rate  of  FAPI=0.30. 

Referring  again  to  Fig.  8,  it  is  noted  that  LLRT-based 
fusion  of  strings  1,2  and  3  provides: 


Fig.  6.  LLRT-based  fusion  results  (train  on  all  data). 


Fig.  7.  Majority  voting  and  LLRT-based  fusion  algorithm 
comparison. 
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•  An  8:1  false  alarm  reduction  over  the  best  performing 
string  (algorithm  1),  at  PjjP^O.93. 

•  An  increased  mine  classification  probability  of 
PdPc=0.977,  at  a  very  low  false  alarm  rate  of  FAPI=0.08. 

•  The  capability  to  detect  85  out  of  86  targets,  at  the  cost 
of  an  increased  false  alarm  rate  of  FAP1=0.64. 

•  An  additional  4.8:1  false  alarm  reduction  vs.  the  string 
that  fuses  2  algorithms,  at  P(|PC=0.93. 

Finally,  referring  again  to  Fig.  8,  it  is  noted  that  LLRT- 
based  fusion  of  all  4  processing  strings  provides: 

•  An  8:1  false  alarm  reduction  over  the  best  performing 
string  (algorithm  1),  at  PC|PC=0.93. 

•  The  capability  to  correctly  call  85  out  of  86  targets,  at  a 
cost  of  slightly  increased  false  alarm  rate  vs.  the  best 
performing  algorithm. 

•  The  capability  to  correctly  call  all  86  targets,  but  at  the 
cost  of  increased  false  alarm  rate  of  FAPI=  1 . 

•  An  additional  2.5:1  false  alarm  reduction  vs.  the  string 
that  fuses  3  algorithms,  at  P(iPc=0.988. 

Thus,  each  added  processing  string  provides  improved 
fusion  performance,  in  terms  of  false  alarm  reduction  and 
improved  target  classification  capability,  even  when  adding 
processing  strings  that  on  the  surface  appear  to  be  much  worst 
performers  than  the  ones  already  being  utilized.  It  should 
also  be  noted  that  performance  of  algorithms  varies  with 
different  datasets.  Thus,  with  a  different  dataset,  algorithms  3 
and  4  could  be  the  top  performers  and  1  and  2  could  be  the 
worst  ones.  Therefore,  only  by  utilizing  as  many  distinct 
algorithms  as  possible,  and  their  fusion,  one  can  attain  robust 
system  performance  in  a  variety  of  environments. 
Additionally,  each  algorithm  developer  is  constantly  striving 
to  improve  performance  with  each  new  sensor  and  datasets  as 
they  become  available. 

IV.  CONCLUSIONS 

A  new,  automated  sea  mine  CAD  /  CAC  processing 
string  has  been  developed  and  successfully  tested  with  real 
high-resolution  sonar  imagery  data,  collected  in  a  shallow 
water  environment.  This  robust  processing  string  involves  the 
fusion  of  the  outputs  of  an  assortment  of  mine  classification 
algorithms.  The  overall  CAD/CAC  processing  string  includes 
pre-processing,  adaptive  clutter  filtering  (ACF), 
normalization,  detection,  feature  extraction,  feature 


Fig.  8.  LLRT-based  fusion  investigation:  performance  gains  provided  by 
fusing  2,  3  or  4  CAD/CAC  strings. 


orthogonalization,  optimal  subset  feature  selection, 
classification  and  fusion  processing  blocks. 

Pre-processing  includes  image  pre-normalization, 
clipping,  data  decimation  and  mean  subtraction.  The  ACF  is 
an  adaptive  linear  FIR  filter  that  is  optimal  in  the  Least 
Squares  sense,  and  is  applied  to  low-resolution  data.  A  range 
dimension  ACF  is  utilized,  that  is  matched  both  to  average 
highlight  and  shadow  information  (normalized  mine  target 
shape,  obtained  by  prior  data  analysis  from  a  training  set), 
while  simultaneously  suppressing  background  clutter. 
Application  of  post-ACF  normalization,  removes  any 
remaining  data  non-stationarities,  whereas  the  following 
detection  consists  of  thresholding,  clustering  of  exceedances 
into  objects  (based  on  the  expected  mine  size),  limiting  the 
number  of  detections  and  rejecting  point-like  objects. 

Subsequently,  features  are  extracted  from  high-resolution 
input  data  and  an  orthogonalization  transformation  is  applied 
to  the  features,  enabling  an  efficient  application  of  the  optimal 
log-likelihood-ratio-test  (LLRT)  classification  rule.  Two 
significant  improvements  were  made  to  the  CAD/CAC 
processing  string  by  employing  sub-image  adaptive  clutter 
filters  (SACF)  and  utilizing  a  repeated  application  of  the 
subset  feature  selection  /  classification  blocks. 

The  utility  of  the  Lockheed  Martin’s  CAD/CAC 
processing  string  and  the  CAD/CAC  processing  string  fusion 
was  demonstrated  with  a  new  shallow  water  data  set.  The 
processing  string  detection  and  classification  parameters  were 
tuned  and  the  string  classification  performance  was 
optimized,  by  appropriately  selecting  a  subset  of  the  original 
feature  set,  at  each  classification  stage  in  the  multi-stage 
design  classification  process.  The  SACF-based  improved 
CAD/CAC  processing  string  resulted  in  increased  target 
detectability,  PC|PC=0.93,  vs.  Pt]Pc=0.9 1  that  was  previously 
obtained  with  the  baseline  single  filter  per  image  ACF-based 
CAD/CAC  processing  string.  The  cascade  classifier  based 
CAD/CAC  processing  string  resulted  in  a  3.6:1  false  alarm 
reduction,  vs.  the  baseline  single  stage  classification  string 
that  was  previously  utilized. 

The  classified  objects  of  four  distinct  processing  strings, 
developed  by  four  different  research  teams,  were  fused  using 
the  classified  objects’  confidence  values  as  features.  The 
fusion  approaches  investigated  included  “M-out-of-N”  and 
LLRT-based  fusion.  Best  performance  was  obtained  by 
utilizing  LLRT-based  fusion,  and  training  the  algorithm  on 
all  the  data.  The  LLRT-based  fusion  resulted  in  an  11:1  false 
alarm  reduction  improvement  over  the  “2-out-of-4”  rule.  A 
“modified  2-out-of-4”  rule  was  also  developed  which 
provided  a  25%  false  alarm  reduction  over  the  baseline  “2- 
out-of-4”  fusion  rule. 

The  overall  CAD/CAC  processing  string  LLRT-based 
fusion  resulted  in  improved  mine  classification  capability, 
providing  up  to  an  eight-fold  false  alarm  rate  reduction, 
compared  to  the  best  single  CAD/CAC  processing  string 
results.  Alternatively,  the  LLRT-based  fusion  of  the 
CAD/CAC  strings  provided  excellent  mine  classification 
performance  (P(|PC=0.988,  with  only  FAPI=0.257).  Even 
when  training  the  LLRT  algorithm  on  a  training  set  only,  it 
still  significantly  outperforms  the  “2-out-of-4”  fusion. 
Therefore,  the  LLRT-based  fusion  algorithm  is  the 
recommended  fusion  approach. 
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A  new  investigation  was  presented  illustrating  the 
performance  gains  provided  by  utilizing  LLRT-based  fusion 
of  2,  3  or  4  processing  strings.  Each  added  string  provides  a 
performance  gain,  in  terms  of  false  alarm  reduction  and/or 
increased  mine  classification  capability,  even  when  adding 
strings  that  appear  to  be  significantly  worst  performers.  Thus 
it  is  recommended,  to  utilize  as  many  distinct  algorithms  as 
possible  and  their  fusion,  aiming  to  attain  robust  overall 
CAD/CAC  system  performance  in  a  variety  of  environments. 

Future  plans  include  evaluating  the  effectiveness  of  the 
overall  SACF,  detection,  feature  extraction  and 
orthogonalization,  FFRT  classification  and  FFRT-based 
fusion  string  with  new  sensor  data,  and  implementing  and 
testing  the  overall  CAD/CAC  and  fusion  processing  string  in 
real-time  on  board  the  UUV. 

This  paper  has  presented  single  CAD/CAC  and  fused 
processing  string  results  with  new  SW  high-resolution  data. 
Although  it  is  very  tempting  to  put  undue  emphasis  on  the 
single  string  that  provides  the  best  classification  results,  one 
must  refrain  from  doing  so  for  a  variety  of  reasons.  First  of 
all,  the  results  presented  here  are  preliminary  and  all  four 
research  teams  are  striving  to  improve  their  CAD/CAC  string 
performance.  Second,  if  a  given  processing  string  provides 
the  best  results  with  a  particular  dataset,  there  is  no  guarantee 
that  it  will  provide  the  best  results  with  a  different  dataset. 
Thus,  the  emphasis  must  be  placed  on  the  performance  gains 
provided  by  fusing  drastically  different  strings,  providing 
improvement  both  in  terms  of  false  alarm  reduction  and/or 
increased  correct  target  classification  capability. 

It  must  also  be  noted  that  the  results  presented  here  were 
obtained  with  a  benign  dataset.  When  recently  processing 
other  datasets,  including  a  newer  extensive  more  difficult 
VSW  dataset,  the  need  to  develop  processing  string 
improvements  has  become  evident. 
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